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ABSTRACT 

This paper describes the Guided Approach to 
Instructional Design Advising (GAIDA) , an automated instructional 
design tool that incorporates techniques of artificial intelligence. 
GAIDA was developed by the U.S. Air Force Armstrong Laboratory to 
facilitate the planning and production of interactive courseware and 
computer-based training materials. The tool is a case-based system 
that incorporates a short exposition of Gagne ' s nine events of 
instruction and four complete examples of applying the nine events to 
identification of naval insignia, classifications of electronic 
resistors, checklist procedure for the F-16 Gatling gun, and 
procedure for testing a patient's respiratory capacity using a 
spirometer. The paper covers (1) the issues involved in developing an 
automated instructional design advisor; (2) the results of Gagne ! s 
evaluation of the initial GAIDA case (the checklist for the F-16 
gun); (3) a description of how GAIDA has been redesigned; and (A) how 
GAIDA will be used in future instructional design research and 
development projects. (Contains 18 references.) (KRN) 
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ABSTRACT 



Interactive courseware (ICW) and computer-based training (CBT) 
development are growing in demand as well as in complexity. Departments of 
instructional technology are simply unable to keep up with the demand. The 
situation is most severe in the Department of Defense (DoD) where declining 
training budgets restrict how much can be spent on contracted efforts 
(typically costly) and where a declining number of military training 
specialists force the DoD to consider alternative means of achieving the 
development of more cost effective ICW/CBT using primarily in-house 
expertise. The solution is to provide automated systems to support the 
development of ICW and CBT. One promising technology in this area is to 
incorporate the techniques of artificial intelligence in automated tools for 
instructional design. This paper describes the rationale of one such system, 
the Guided Approach to Instructional Design Advising or GAIDA, which 
probably represents the low end of the scale in artificial intelligence training 
support systems. 



SUMMARY 

The Air Force Armstrong Laboratory is exploring automated means for assisting 
instructional designers in the planning and production of computer-based technical training 
materials (interactive courseware, ICW). The motivation for this effort is twofold: (1) 
increasing c smands for computer-based (CBT) training materials, and (2) limited computer- 
based instru tional design expertise. Our major research and development effort in this 
domain is ca,!ed the Advanced Instructional Design Advisor (AIDA) (Muraida & Spector, 
1990). AIDA represents a systematic attempt to encapsulate established instructional 
strategy frameworks appropriate to a selected but substantial subset of instructional 
objectives and to make them available to subject-matter experts for content input. The 
AIDA effort has produced two interesting instructional design tools. The first tool focuses 
on objective-specific design expertise and incorporates expert system technology in the form 
of rules for configuring specific instructional interactions. The second tool incorporates 
case-based reasoning methodologies and provides completely worked examples of four 
technical training modules representing a variety of training situations. 

The first tool (XAIDA or Experimental Advanced Instructional Design Advisor) uses 
object-oriented instructional frameworks that represent an implementation of second 
generation instructional design and the four associated transaction shells (Merrill, Li, & 
Jones, 1990). XAIDA is currently undergoing formative evaluation at a field site. The next 
phase of research for XAIDA will include systematically varying instructional parameters 
in order to develop empirically proven instructional tactics. For example, a transaction can 
be configured so as to provide by default a given amount and type of performance 
feedback at a given point in a particular type of lesson. Lessons can then be quickly 
developed, delivered to students, and evaluated with regard to effectiveness. This 
capability will allow data to be collected to determine optimal instructional tactics, given a 
particular type of instructional objective and overall strategy. 

The second tool (GAIDA or Guided Approach to Instructional Design Advising) is 
based explicitly on Gagn6*s nine events of instruction (1985) and is the focus of this paper. 
GAIDA is a case-based system that incorporates a short exposition of the nine events of 
instruction, as well as four complete examples of applying the nine events to identification 
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of naval insignia, classification of electronic resistors, checklist procedure for the F-16 
gatling gun, and procedure for testing a patient's respiratory capacity using a spirometer. 
The examples can be displayed from two perspectives: (1) instructional designer, and (2) 
student. In student mode GAIDA allows the user to select one of these four lessons and then 
presents a computer-based training session including performance measures. When in 
instructional designer mode GAIDA allows the user to select a lesson and view specific 
design guidance (an elaboration of the nine events appropriate for that type of lesson 
objective) along with an abbreviated representation of the actual lesson. The designer can 
jump to student mode at any point in the elaboration to get a closer view of the lesson. 



OBJECTIVES 

This paper will include a discussion of the research program in automating 
instructional design that resulted in the GAIDA software. Specific purposes include the 
following: 

(1) Discuss the issues involved in developing an automated instructional design 
advisor, focusing on offering online guidance for courseware development; 

(2) Report the results of GagnS's evaluation of his initial GAIDA case, the checklist 
procedure for the F-16 gun (Gagnd, 1992); 

(3) Describe how GAIDA has been redesigned and demonstrate all four cases; and 

(4) Indicate how GAIDA will be used in future instructional design research and 
development projects. 



BACKGROUND 

Instructional Systems Development (ISP) 

Most ISD models are based on an engineering approach to curriculum and lesson 
development (Andrews & Goodson, 1980). These models typically divide the ISD process 
into five stages or phases: (1) analysis, (2) design, (3) production, (4) implementation, and 
(5) maintenance (Golas & Shriver, 1991). Associated with each phase are specific goals and 
subtasks. For example, the design phase is concerned with specifying instructional 
objectives, grouping and sequencing those objectives, and identifying appropriate 
instructional treatments. 

Older ISD models often fail to account for relevant cognitive aspects of the learning 
task (Tennyson, 1991). For example, a behaviorally oriented ISD model might suggest that 
the grouping and sequencing of instructional objectives should always reflect the specific 
sequences of tasks actually performed on the job. While this is certainly true in some cases, 
there are cases when a cognitively oriented ISD model would suggest that it is more 
effective in promoting recall, retention, and transfer to group and sequence objectives 
around relevant mental models. In addition, older ISD models have suggested that the 
activities in each phase are sequential and that evaluation comes toward the end of the 
process. A recent study of expert instructional designers indicates that they seldom perform 
the idealized activities of an ISD model in a sequential manner (Rowland, 1992). Moreover, 
it is appropriate to include evaluation (quality control and both formative and summative 
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evaluation) into each phase and subtask of the model (Tennyson, 1991). 

The value of an ISD model is not that it prescribes a definite set of activities to be 
performed in a definite order so as to accomplish an effective instructional development. 
The real value of an ISD model is that it provides a meaningful organizing framework 
within which development activities can be described, discussed, and possibly actualized. 
In other words, an ISD model is a kind of mental model for instructional development, and 
it should be regarded as such, rather than as a rigid set of task prescriptions. In any event, 
for the purposes of this paper, the ISD model sketched above will provide a framework for 
the applicability of various AI techniques. 

Courseware Desig n 

With the advent of affordable personal computers and increasing capabilities to 
support multimedia, the process of developing instruction targeted for delivery by 
computers (courseware) has caused ISD models to accommodate the special requirements of 
CBT and ICW. For example, continuing interaction with programming and systems 
specialists is now planned for most serious courseware development efforts. 

However, it is widely recognized that there is a shortage of well-trained and 
experienced ICW developers (Spector, Muraida, & Dallman, 1990), In addition, there exists 
incomplete knowledge concerning how to optimize various media in support of specific 
learning objectives (Friedman, Poison, & Spector, 1991). However, it is also commonly 
acknowledged that careful planning is essential for the successful development of 
courseware. Unlike instruction to be delivered by humans, ICW must be robust and capable 
of being made adaptable or generalizable across a variety of students and settings. Efforts 
to develop ISD models and standards specifically for ICW are now commonplace, both in the 
military as well as in industry (e.g., see MIL-STD-1379D, Appendix B, Systems Approach to 
the Interactive Courseware Development Process). 

Technical Training in the Department of Defense (DoD) 

The shortage of ICW experts is particularly acute in the DoD. Typically, Ihc 
military instructional designer is a subject-matter expert (SME) who has completed a brief 
course on designing instruction (Spector et a/., 1990). For example, the technical training 
specialist may be a non-commissioned officer with a great deal of experience with a 
particular piece of equipment but with very little background in educational psychology or 
instructional technology. 

With declining training budgets and personnel cutbacks it appears unlikely that the 
DoD will be able to accomplish all of its technical training ICW requirements through 
outside contracts. However, there will continue to be requirements for effective technical 
trainin r in the military. In fact, given the impetus towards a smaller but more flexible 
force structure using highly complex weaponry, it is most likely that technical training 
requirements will remain critical. In order to satisfy these requirements, there willl be a 
demand for increasing use of ICW and for increasingly effective ICW. As a consequence, 
DoD laboratories have been exploring ways to provide the required ICW expertise to the 
SMEs responsible for the development of much of DoD courseware. As the center of 
excellence for artificial intelligence (AI) in training, Armstrong Laboratory is especially 
interested in applying AI techniques to the design of effective ICW. 



Artificial Intelligence & Training 



In order to complete the context for the system described below, it is necessary to 
provide a working definition of AI. There are, of course, many definitions in the AI 
literature. Some emphasize the psychological aspects of human intelligence and various 
methods of modeling human intelligence. Others emphasize the mathematical complexity of 
certain problems addressed by AI researchers. A more neutral approach is taken by Rich 
and Knight who define AI to be "the study of how to make computers do things which, at 
the moment, people do better" (1991, p. 3). It is not clear what benefit is to be derived from 
the endless debates concerning whether particular machines and systems exhibit 
intelligence. So as to avoid those debates, I shall adopt Gagn6's suggestion that AI be 
regarded simply as engineered cognition (Spector, Gagn6, Muraida, & Dimitroff, 1992). 

The most prevalent AI architectures are expert systems and artificial neural 
networks. To date there has been very little work in applying artificial neural network 
technology to training systems. The most obvious application for neural networks is in 
delivery systems involving speech recognition and/or generation, but there has been very 
little development of such systems for training purposes. As a consequence, the most used 
AI technology in training is the expert system, especially those using planning architectures. 

Expert Systems 

For the sake of this discussion, an expert system is an AI system which consists of a 
rule base, an inference engine, a database, and an interface with a user. The rule base is 
usually represented in the form of IF-THEN statements and is taken to represent the 
knowledge of an expert in the subject. The inference engine provides a mechanism to 
examine the current status of the system, search the rule base, find any and all applicable 
rules, select an appropriate rule to apply, and then apply the rule, thereby creating a new 
system status. It should be obvious that there is nothing magical or mysterious about expert 
systems. They do, however, represent a very powerful problem solving technique. 

Expert systems arc normally characterized as either planning systems or as 
diagnostic systems. Planning systems work forward from a given system state toward a 
final state. For example, a planning system could take input regarding lesson objectives and 
subject matter and develop a lesson outline, a lesson plan, and possibly even a prototype 
lesson. On the other hand, diagnostic systems work backward from a given system state 
toward a cause or analysis. For example, a diagnostic system could take a given ICW lesson 
and student performance data as input and provide an explanation why certain parts of the 
lesson were not working well. There have been no diagnostic expert systems developed and 
tested for the domain of instructional design, although Duchastel has recommended that 
this is a possibility worth exploring (1990). 

Planning Architectures 

There have been a number of attempts to develop expert planning systems to support 
various aspects of the ISD process. Most notable among these efforts is Merrill's ID Expert 
(Merrill, Li, & Jones, 1990), which provided the core technology (transaction theory) for 
AIDA. The original motivation behind ID Expert was to develop a rule base of 
instructional prescriptions and to use those rules to generate appropriate frameworks for 
lesson materials which would be provided by an SME. Other applications of expert 
planning systems to various ISD processes are possible. 
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Table 1 provides an organizing framework for the various ways that AI has been 
incorporated into the ISD process. The Guided Approach to Instructional Design Advising 
(GAIDA) is based explicitly on GagnS's nine events of instruction (Gagnd, 1985). 



Table 1 



AI Techni q ue^ ISD 



I. Analysis 

Training Requirements 



II, Design 

Instructional Design 



Tools 

Cognitive Task Analysis 



Advisors 
Online Examples & 

Guidelines 
Case-based Systems 



Decision Support Systems 



Rule-based Systems 
Intelligent Tutors & 
& Critiquers 



III. Production 
Intelligent Development 



IV. Implementation 



Adaptive Delivery 
Intelligent Tutors 
Adaptive Testing 



Tools 

Graphics Mini-advisors 



Audio, Video, and 

Interface Tools 

Intelligent Lesson 
Templates 
(e.g., XAIDA) 



These examples of applicable AI techniques in the ISD domain are not meant to be 
exhaustive. What is offered here is a way to classify various efforts to develop intelligent 
instructional systems. It is worth noticing that no examples of intelligent applications in 
the maintenance phase are identified, although it is possible to imagine an automated 
instructional system that monitored either the instructional development process or the 
progress of learners using the system, processed the results, and filtered those results 
through a set of rules which prescribed variations on the development or delivery process. 

For example, a system could record and analyze answers to questions. If a particular 
question was never answered correctly, the system might recommend a remedy for the 
situation. A system might also monitor where learners spent most of their time with the 
system and analyze how that time contributed to learning outcomes. If it appeared that 
time was wasted in one part of the system, then the system might recommend some kind of 



V. Maintenance 
Monitoring/Diagnostic Tools 
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remedy for that situation. 



There is nothing in this framework to prevent a particular system from being 
categorized in more than one area. In fact, we view XAIDA as an intelligent application in 
the middle three phases of ISD, although our research interests are clearly focused on the 
design phase. 

Approaches to Automating Instructional Design (ID) 

As already mentioned the two projects mentioned here focus on the application of AI 
to the instructional design phase- It is worth noting that GAIDA and XAIDA represent 
only two of a number of possible approaches to intelligent instructional design. Tennyson 
has proposed the more ambitious approach of building an intelligent tutoring system (ITS) 
for the domain of instructional design (1991). Duchastel has proposed an expert critiquing 
system which can evaluate designs created by relatively experienced instructional designers 
(1990). It is probably premature to proceed with such ambitious efforts, however. Indeed, 
the validated results of efforts like GAIDA and XAIDA will form an essential part of an 
expert ISD critiquer or an ITS for ID. 



THEORETICAL FRAMEWORK FOR GAIDA 

Applications for Courseware Design 

Online Guidance 

The most direct method of capturing instructional design expertise and providing it 
to novice designers is to construct an expert tutorial on instructional design and to connect 
it to various steps in the development process. This is the basic idea behind the ID Advisor 
(idEa) developed by Progressive Learning Systems and Ford Aerospace's TIPS (The 
Instructional Prescription System). Because novice designers may desire the rigors of a 
short course or tutorial in the course of their experiences, providing such guidance online 
and in the context of their design efforts appears highly desirable and is included in the 
development plan for XAIDA. 

Elaborated Examples 

A case-based approach to instructional design most closely approximates what expert 
designers actually do (Rowland, 1992). GAIDA uses this approach to provide four 
completely worked examples of applying Gagni's nine events of instruction to technical 
training lessons in a CBT environment. The four examples involve an identification task, a 
classification task, a checklist procedure, and a memory procedure. Subject material ranges 
from identifying naval insignia, classifying resistors, checking the operation of the F-16 
gatling gun, and interpreting the results of a medical procedure. 

Design Implications for GAIDA 

Three components comprise the theoretical framework for GAIDA: (1) different 
learning objectives require different instructional strategies (Gagnd, 1992), (2) instruction 
consists of a set of events (usually nine) external to the learner which are designed to 
support the internal processes of learning, and (3) elaboration of these events varies with 
the learning objective (verbal knowledge, concepts, procedural rules, motor skills, and 
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attitudes) and with the learning env ; -onment (determined by both delivery and learner 
capabilities). 1 



METHODOLOGY IN DEVELOPING GAIDA 

GAIDA is coded in ToolBook 1.5, a hypermedia authoring tool that runs under 
Windows 3.0 on Intel 80386/80486 personal computers. Professor Gagn6 provided system 
specifications as part of his work at the Armstrong Laboratory as a National Research 
Fellow. The programming has been accomplished by graduate students and by Armstrong 
Laboratory programmers. 

GAIDA was originally conceived as an alternative to the XAIDA system (Gagnd, 
Tennyson, & Gettman, 1991). GAIDA is designed to leave detailed implementation of the 
instructional design to the user. In contrast, XAIDA is designed to relieve the user of as 
much instructional design decision making as possible. A comparative analysis of GAIDA 
and XAIDA is planned next year after both systems have been revised and retested in 
accordance with their respective formative evaluations. 

GagnS's formative evaluation was conducted at the Lowry Air Force Base Technical 
Training Center (1992). Six subjects participated in the study. Each subject was acquainted 
with the subject matter; five were novice instructional designers; one was generally familiar 
with instructional design. Subjects were presented with the GAIDA software which 
described how to apply the nine events of instruction to teach the procedure for a ground 
check of the F-16 gun. Subjects were asked to follow the design guidance being suggested 
by the software and create a paper-based version of a similar lesson. Subjects kept detailed 
notebooks on their reactions and were asked to think aloud at key points concerning three 
criteria: clarity, impact, and feasibility. Results of this evaluation were quite favorable 
(Gagn6, 1992). Subjects generally found the design guidance understandable, meaningful, 
and useful. 

The experienced designer took exception with the stated purpose of the lesson and 
argued that what students needed to be taught was more system knowledge. This finding is 
consistent with Rowland's finding that experienced designers spent much of their initial 
lesson planning critically examining the stated purpose of the instruction (1992). Rowland 
also noticed that experienced designers apparently worked from the perspective of similar 
cases. This finding suggests that the GAIDA approach of providing less experienced 
designers with worked cases annotated with a detailed instructional design commentary is 
likely to enhance the quality of CBT produced by novice designers. 



CONCLUSIONS 
GAIDA's Utility 

GAIDA has now been tested by more than a dozen novice designers, all of whom 
have found it to provide useful and meaningful instructional design guidance. As a 
consequence, it is being provided to users as an interim product to supplement the 
instructional design training provided by the Air Force. GAIDA is extensible in the sense 
that users can add their own best cases by adhering to a few ToolBook conventions when 
creating the sample lesson. 
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Unresolved Issues 



There is much that wc do not know concerning how individuals learn, and, as a 
consequence, how to support learning processes. There is reason to expect that collaborative 
learning will be highly effective in certain computer-based environments (Stephenson, 
1991). Wc also expect to find that auditory presentations will be highly effective in 
particular learning situations (Spector & Muraida, 1991). However, the specifics concerning 
these and other factors relevant to the design of a more intelligent advisor for ICW design 
await further empirical research. 

Having established the utility of GAIDA in the domain of instructional design 
advising for novice ICW developers, we are now in a position to use GAIDA and the 
associated data as a baseline of comparison to how useful more sophisticated instructional 
designer advising systems are (e.g., when we evaluate XAIDA we can easily create treatment 
groups which receive GAIDA advising in addition to or in place of XAIDA's intelligent 
framework). 
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